Systems and methods for selecting spheres of relevance for presenting augmented reality information

ABSTRACT

Augmented reality information may be presented to a user without overloading the user with an extraordinary amount of information. AR information may be selectively presented to the user in iterative ranges from the user in the user&#39;s field of view. Real-world objects in the user&#39;s field of view may be detected. A first group of objects may be selected that are less than a first distance from the user in the field of view, and a second group of objects selected that are between the first distance and a second greater distance in the field of view. At a first time, the first group of objects may be augmented to the user. At a second time after the first time, the second group of objects may be augmented to the user. The first group may stop being augmented prior to the second time.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is continuation of U.S. patent application Ser.No. 16/466,611, filed Jun. 4, 2019, which is a national stageapplication under 35 U.S.C. 371 of International Application No.PCT/US2017/66472, entitled SYSTEMS AND METHODS FOR SELECTING SPHERES OFRELEVANCE FOR PRESENTING AUGMENTED REALITY INFORMATION, filed on Dec.14, 2017, which claims benefit under 35 U.S.C. § 119(e) from U.S.Provisional Patent Application Ser. No. 62/437,458, filed Dec. 21, 2016,entitled “SYSTEMS AND METHODS FOR SELECTING SPHERES OF RELEVANCE FORPRESENTING AUGMENTED REALITY INFORMATION”, which are incorporated hereinby reference in its entirety.

TECHNICAL FIELD

This disclosure relates to systems and methods for augmented realitydevices. More specifically, this disclosure relates to systems andmethods for managing presentation of augmented reality content to usersof augmented reality devices.

BACKGROUND

Augmented reality (AR) is an emerging technology that allows users toaugment their perception of the world, through use of AR head-mounteddisplays (HMDs), both transparent and opaque, display screens, andvarious other types of devices. Current AR approaches provide ARinformation about real-world objects that are in a user's field of view,including objects that may be too far away to be discerned by the user.Such approaches may overwhelm the user with AR information, especiallyin environments where there are many real-world objects in the user'sfield of view for which information is available for presentation to theuser. After a while, such information can become less and less relevantto the user. In some upcoming AR approaches, only information aboutreal-world objects on which the user's gaze rests is provided. However,although such approaches provide AR information about fewer real-worldobjects, they select the real-world objects to be only those that theuser's gaze rests on. This would work only for objects that aresufficiently close to the user for the user's gaze to be recognized asresting upon them. Additionally, the gaze-based approaches call for auser to take the initiative and thus do not bring unanticipated orunknown information to the user. For example, if the user is not awarethat a concert will be held in a nondescript park nearby, the user wouldnot gaze at the park long enough to be presented that information.

SUMMARY

Systems and methods set forth herein inform a user of information thatmay not be in the user's immediate gaze, without overloading the userwith an extraordinary amount of information. Embodiments describedherein inform the user in a way that is helpful but not intrusive anddoes not require the user to take the initiative. Further, embodimentsdescribed herein inform the user in a way that reduces interference withthe scene and streamlines the presentation of information so that theuser's attention and cognition are not overloaded.

In some embodiments, systems and methods are described for selectingaugmented reality information about one or more real-world objects fordisplay in a user's view comprising, iteratively, receiving one or morepairs of information, each pair comprising (a) one real-world objectidentified by its geospatial coordinates, and (b) its associated ARobject (that specifies and renders some associated augmented informationin a suitable manner, including the name and other attributes of thereal-world object), determining two or more spheres of relevance ofincreasing radii and centered on the user, determining the relevantsphere for current presentation to the user, and displaying AR objectsfrom among the received real-world objects whose coordinates lie withinthe current relevant sphere and within no inner sphere and notdisplaying any other AR objects.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding may be had from the following description,presented by way of example in conjunction with the accompanyingdrawings in which like reference numerals in the figures indicate likeelements, and wherein:

FIG. 1A depicts an example communications system in which one or moredisclosed embodiments may be implemented.

FIG. 1B depicts an example wireless transmit/receive unit (WTRU) thatmay be used within the communications system of FIG. 1A.

FIG. 1C depicts an exemplary network entity that may be used within thecommunication system of FIG. 1A.

FIG. 2 is a block diagram of a possible architecture, in accordance withsome embodiments.

FIG. 3 is a flowchart of a process, in accordance with some embodiments.

FIG. 4 is a call-flow diagram of a process, in accordance with someembodiments.

FIGS. 5A-5D illustrate sphere progression for an AR device, inaccordance with some embodiments.

FIGS. 6A-6E illustrate AR information presentation progression, inaccordance with some embodiments.

FIG. 7 illustrates a progression of augmenting real-world objects, inaccordance with some embodiments.

EXAMPLE NETWORKS FOR IMPLEMENTATION OF THE EMBODIMENTS

FIG. 1A is a diagram illustrating an example communications system 100in which one or more disclosed embodiments may be implemented. Thecommunications system 100 may be a multiple access system that providescontent, such as voice, data, video, messaging, broadcast, etc., tomultiple wireless users. The communications system 100 may enablemultiple wireless users to access such content through the sharing ofsystem resources, including wireless bandwidth. For example, thecommunications systems 100 may employ one or more channel accessmethods, such as code division multiple access (CDMA), time divisionmultiple access (TDMA), frequency division multiple access (FDMA),orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tailunique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM(UW-OFDM), resource block-filtered OFDM, filter bank multicarrier(FBMC), and the like.

The systems and methods relating to efficient use of AR devices may beused with the wireless communication systems described with respect toFIGS. 1A-1C. As an initial matter, these wireless systems will bedescribed. FIG. 1A is a diagram of an example communications system 100in which one or more disclosed embodiments may be implemented. Thecommunications system 100 may be a multiple access system that providescontent, such as voice, data, video, messaging, broadcast, and the like,to multiple wireless users. The communications system 100 may enablemultiple wireless users to access such content through the sharing ofsystem resources, including wireless bandwidth.

As shown in FIG. 1A, the communications system 100 may include WTRUs 102a, 102 b, 102 c, and/or 102 d (which generally or collectively may bereferred to as WTRU 102), a RAN 103/104/105, a core network 106/107/109,a public switched telephone network (PSTN) 108, the Internet 110, andother networks 112, though it will be appreciated that the disclosedembodiments contemplate any number of WTRUs, base stations, networks,and/or network elements. Each of the WTRUs 102 a, 102 b, 102 c, 102 dmay be any type of device configured to operate and/or communicate in awireless environment. By way of example, the WTRUs 102 a, 102 b, 102 c,102 d may be configured to transmit and/or receive wireless signals andmay include user equipment (UE), a mobile station, a fixed or mobilesubscriber unit, a pager, a cellular telephone, a personal digitalassistant (PDA), a smartphone, a laptop, a netbook, a personal computer,a wireless sensor, consumer electronics, and the like.

The communications systems 100 may also include a base station 114 a anda base station 114 b. Each of the base stations 114 a, 114 b may be anytype of device configured to wirelessly interface with at least one ofthe WTRUs 102 a, 102 b, 102 c, 102 d to facilitate access to one or morecommunication networks, such as the core network 106/107/109, theInternet 110, and/or the networks 112. By way of example, the basestations 114 a, 114 b may be a base transceiver station (BTS), a Node-B,an eNode B, a Home Node B, a Home eNode B, a site controller, an accesspoint (AP), a wireless router, and the like. While the base stations 114a, 114 b are each depicted as a single element, it will be appreciatedthat the base stations 114 a, 114 b may include any number ofinterconnected base stations and/or network elements.

The base station 114 a may be part of the RAN 103/104/105, which mayalso include other base stations and/or network elements (not shown),such as a base station controller (BSC), a radio network controller(RNC), relay nodes, and the like. The base station 114 a and/or the basestation 114 b may be configured to transmit and/or receive wirelesssignals within a particular geographic region, which may be referred toas a cell (not shown). The cell may further be divided into sectors. Forexample, the cell associated with the base station 114 a may be dividedinto three sectors. Thus, in one embodiment, the base station 114 a mayinclude three transceivers (one for each sector of the cell). In anotherembodiment, the base station 114 a may employ multiple-input multipleoutput (MIMO) technology and, therefore, may utilize multipletransceivers for each sector of the cell.

The base stations 114 a, 114 b may communicate with one or more of theWTRUs 102 a, 102 b, 102 c, 102 d over an air interface 115/116/117,which may be any suitable wireless communication link (e.g., radiofrequency (RF), microwave, infrared (IR), ultraviolet (UV), visiblelight, and the like). The air interface 115/116/117 may be establishedusing any suitable radio access technology (RAT).

More specifically, as noted above, the communications system 100 may bea multiple access system and may employ one or more channel-accessschemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. Forexample, the base station 114 a in the RAN 103/104/105 and the WTRUs 102a, 102 b, 102 c may implement a radio technology such as UniversalMobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA),which may establish the air interface 115/116/117 using wideband CDMA(WCDMA). WCDMA may include communication protocols such as High-SpeedPacket Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may includeHigh-Speed Downlink Packet Access (HSDPA) and/or High-Speed UplinkPacket Access (HSUPA).

In another embodiment, the base station 114 a and the WTRUs 102 a, 102b, 102 c may implement a radio technology such as Evolved UMTSTerrestrial Radio Access (E UTRA), which may establish the air interface115/116/117 using Long Term Evolution (LTE) and/or LTE Advanced (LTE A).

In other embodiments, the base station 114 a and the WTRUs 102 a, 102 b,102 c may implement radio technologies such as IEEE 802.16 (i.e.,Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000,CDMA2000 1×, CDMA2000 EV-DO, Interim Standard 2000 (IS 2000), InterimStandard 95 (IS 95), Interim Standard 856 (IS 856), Global System forMobile communications (GSM), Enhanced Data rates for GSM Evolution(EDGE), GSM EDGE (GERAN), and the like.

The base station 114 b in FIG. 1A may be a wireless router, Home Node B,Home eNode B, or access point, as examples, and may utilize any suitableRAT for facilitating wireless connectivity in a localized area, such asa place of business, a home, a vehicle, a campus, and the like. In oneembodiment, the base station 114 b and the WTRUs 102 c, 102 d mayimplement a radio technology such as IEEE 802.11 to establish a wirelesslocal area network (WLAN). In another embodiment, the base station 114 band the WTRUs 102 c, 102 d may implement a radio technology such as IEEE802.15 to establish a wireless personal area network (WPAN). In yetanother embodiment, the base station 114 b and the WTRUs 102 c, 102 dmay utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE,LTE-A, and the like) to establish a picocell or femtocell. As shown inFIG. 1A, the base station 114 b may have a direct connection to theInternet 110. Thus, the base station 114 b may not be required to accessthe Internet 110 via the core network 106/107/109.

The RAN 103/104/105 may be in communication with the core network106/107/109, which may be any type of network configured to providevoice, data, applications, and/or voice over internet protocol (VoIP)services to one or more of the WTRUs 102 a, 102 b, 102 c, 102 d. Asexamples, the core network 106/107/109 may provide call control, billingservices, mobile location-based services, pre-paid calling, Internetconnectivity, video distribution, and the like, and/or performhigh-level security functions, such as user authentication. Although notshown in FIG. 1A, it will be appreciated that the RAN 103/104/105 and/orthe core network 106/107/109 may be in direct or indirect communicationwith other RANs that employ the same RAT as the RAN 103/104/105 or adifferent RAT. For example, in addition to being connected to the RAN103/104/105, which may be utilizing an E-UTRA radio technology, the corenetwork 106/107/109 may also be in communication with another RAN (notshown) employing a GSM radio technology.

The core network 106/107/109 may also serve as a gateway for the WTRUs102 a, 102 b, 102 c, 102 d to access the PSTN 108, the Internet 110,and/or other networks 112. The PSTN 108 may include circuit-switchedtelephone networks that provide plain old telephone service (POTS). TheInternet 110 may include a global system of interconnected computernetworks and devices that use common communication protocols, such asthe transmission control protocol (TCP), user datagram protocol (UDP)and IP in the TCP/IP Internet protocol suite. The networks 112 mayinclude wired and/or wireless communications networks owned and/oroperated by other service providers. For example, the networks 112 mayinclude another core network connected to one or more RANs, which mayemploy the same RAT as the RAN 103/104/105 or a different RAT.

Some or all of the WTRUs 102 a, 102 b, 102 c, 102 d in thecommunications system 100 may include multi-mode capabilities, i.e., theWTRUs 102 a, 102 b, 102 c, 102 d may include multiple transceivers forcommunicating with different wireless networks over different wirelesslinks. For example, the WTRU 102 c shown in FIG. 1A may be configured tocommunicate with the base station 114 a, which may employ acellular-based radio technology, and with the base station 114 b, whichmay employ an IEEE 802 radio technology.

FIG. 1B is a system diagram of an example WTRU 102. As shown in FIG. 1B,the WTRU 102 may include a processor 118, a transceiver 120, atransmit/receive element 122, a speaker/microphone 124, a keypad 126, adisplay/touchpad 128, a non-removable memory 130, a removable memory132, a power source 134, a global positioning system (GPS) chipset 136,and other peripherals 138. The transceiver 120 may be implemented as acomponent of decoder logic 119. For example, the transceiver 120 anddecoder logic 119 may be implemented on a single LTE or LTE-A chip. Thedecoder logic may include a processor operative to perform instructionsstored in a non-transitory computer-readable medium. As an alternative,or in addition, the decoder logic may be implemented using custom and/orprogrammable digital logic circuitry.

It will be appreciated that the WTRU 102 may include any sub-combinationof the foregoing elements while remaining consistent with an embodiment.Also, embodiments contemplate that the base stations 114 a and 114 b,and/or the nodes that base stations 114 a and 114 b may represent, suchas but not limited to transceiver station (BTS), a Node-B, a sitecontroller, an access point (AP), a home node-B, an evolved home node-B(eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway,and proxy nodes, among others, may include some or all the elementsdepicted in FIG. 1B and described herein.

The processor 118 may be a general-purpose processor, a special purposeprocessor, a conventional processor, a digital signal processor (DSP), aplurality of microprocessors, one or more microprocessors in associationwith a DSP core, a controller, a microcontroller, Application SpecificIntegrated Circuits (ASICs), Field Programmable Gate Array (FPGAs)circuits, any other type of integrated circuit (IC), a state machine,and the like. The processor 118 may perform signal coding, dataprocessing, power control, input/output processing, and/or any otherfunctionality that enables the WTRU 102 to operate in a wirelessenvironment. The processor 118 may be coupled to the transceiver 120,which may be coupled to the transmit/receive element 122. While FIG. 1Bdepicts the processor 118 and the transceiver 120 as separatecomponents, it will be appreciated that the processor 118 and thetransceiver 120 may be integrated together in an electronic package orchip.

The transmit/receive element 122 may be configured to transmit signalsto, or receive signals from, a base station (e.g., the base station 114a) over the air interface 115/116/117. For example, in one embodiment,the transmit/receive element 122 may be an antenna configured totransmit and/or receive RF signals. In another embodiment, thetransmit/receive element 122 may be an emitter/detector configured totransmit and/or receive IR, UV, or visible light signals, as examples.In yet another embodiment, the transmit/receive element 122 may beconfigured to transmit and receive both RF and light signals. It will beappreciated that the transmit/receive element 122 may be configured totransmit and/or receive any combination of wireless signals.

In addition, although the transmit/receive element 122 is depicted inFIG. 1B as a single element, the WTRU 102 may include any number oftransmit/receive elements 122. More specifically, the WTRU 102 mayemploy MIMO technology. Thus, in one embodiment, the WTRU 102 mayinclude two or more transmit/receive elements 122 (e.g., multipleantennas) for transmitting and receiving wireless signals over the airinterface 115/116/117.

The transceiver 120 may be configured to modulate the signals that areto be transmitted by the transmit/receive element 122 and to demodulatethe signals that are received by the transmit/receive element 122. Asnoted above, the WTRU 102 may have multi-mode capabilities. Thus, thetransceiver 120 may include multiple transceivers for enabling the WTRU102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, asexamples.

The processor 118 of the WTRU 102 may be coupled to, and may receiveuser input data from, the speaker/microphone 124, the keypad 126, and/orthe display/touchpad 128 (e.g., a liquid crystal display (LCD) displayunit or organic light-emitting diode (OLED) display unit). The processor118 may also output user data to the speaker/microphone 124, the keypad126, and/or the display/touchpad 128. In addition, the processor 118 mayaccess information from, and store data in, any type of suitable memory,such as the non-removable memory 130 and/or the removable memory 132.The non-removable memory 130 may include random-access memory (RAM),read-only memory (ROM), a hard disk, or any other type of memory storagedevice. The removable memory 132 may include a subscriber identitymodule (SIM) card, a memory stick, a secure digital (SD) memory card,and the like. In other embodiments, the processor 118 may accessinformation from, and store data in, memory that is not physicallylocated on the WTRU 102, such as on a server or a home computer (notshown).

The processor 118 may receive power from the power source 134, and maybe configured to distribute and/or control the power to the othercomponents in the WTRU 102. The power source 134 may be any suitabledevice for powering the WTRU 102. As examples, the power source 134 mayinclude one or more dry cell batteries (e.g., nickel-cadmium (NiCd),nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion),and the like), solar cells, fuel cells, and the like.

The processor 118 may also be coupled to the GPS chipset 136, which maybe configured to provide location information (e.g., longitude andlatitude) regarding the current location of the WTRU 102. In additionto, or in lieu of, the information from the GPS chipset 136, the WTRU102 may receive location information over the air interface 115/116/117from a base station (e.g., base stations 114 a, 114 b) and/or determineits location based on the timing of the signals being received from twoor more nearby base stations. It will be appreciated that the WTRU 102may acquire location information by way of any suitablelocation-determination method while remaining consistent with anembodiment.

The processor 118 may further be coupled to other peripherals 138, whichmay include one or more software and/or hardware modules that provideadditional features, functionality and/or wired or wirelessconnectivity. For example, the peripherals 138 may include anaccelerometer, an e-compass, a satellite transceiver, a digital camera(for photographs or video), a universal serial bus (USB) port, avibration device, a television transceiver, a hands-free headset, aBluetooth® module, a frequency modulated (FM) radio unit, a digitalmusic player, a media player, a video game player module, an Internetbrowser, and the like.

FIG. 1C depicts an example network entity 190 that may be used withinthe communication system 100 of FIG. 1A. As depicted in FIG. 1C, networkentity 190 includes a communication interface 192, a processor 194, andnon-transitory data storage 196, all of which are communicatively linkedby a bus, network, or other communication path 198.

Communication interface 192 may include one or more wired communicationinterfaces and/or one or more wireless-communication interfaces. Withrespect to wired communication, communication interface 192 may includeone or more interfaces such as Ethernet interfaces, as an example. Withrespect to wireless communication, communication interface 192 mayinclude components such as one or more antennae, one or moretransceivers/chipsets designed and configured for one or more types ofwireless (e.g., LTE) communication, and/or any other components deemedsuitable by those of skill in the relevant art. And further with respectto wireless communication, communication interface 192 may be equippedat a scale and with a configuration appropriate for acting on thenetwork side—as opposed to the client side—of wireless communications(e.g., LTE communications, Wi Fi communications, and the like). Thus,communication interface 192 may include the appropriate equipment andcircuitry (perhaps including multiple transceivers) for serving multiplemobile stations, UEs, or other access terminals in a coverage area.

Processor 194 may include one or more processors of any type deemedsuitable by those of skill in the relevant art, some examples includinga general-purpose microprocessor and a dedicated DSP.

Data storage 196 may take the form of any non-transitorycomputer-readable medium or combination of such media, some examplesincluding flash memory, read-only memory (ROM), and random-access memory(RAM) to name but a few, as any one or more types of non-transitory datastorage deemed suitable by those of skill in the relevant art could beused. As depicted in FIG. 1C, data storage 196 contains programinstructions 197 executable by processor 194 for carrying out variouscombinations of the various network-entity functions described herein.

In some embodiments, the network-entity functions described herein arecarried out by a network entity having a structure similar to that ofnetwork entity 190 of FIG. 1C. In some embodiments, one or more of suchfunctions are carried out by a set of multiple network entities incombination, where each network entity has a structure similar to thatof network entity 190 of FIG. 1C. In various different embodiments,network entity 190 is—or at least includes—one or more of (one or moreentities in) RAN 103, (one or more entities in) RAN 104, (one or moreentities in) RAN 105, (one or more entities in) core network 106, (oneor more entities in) core network 107, (one or more entities in) corenetwork 109, base station 114 a, base station 114 b, Node B 140 a, NodeB 140 b, Node B 140 c, RNC 142 a, RNC 142 b, MGW 144, MSC 146, SGSN 148,GGSN 150, eNode B 160 a, eNode B 160 b, eNode B 160 c, MME 162, servinggateway 164, PDN gateway 166, base station 180 a, base station 180 b,base station 180 c, ASN gateway 182, MIP HA 184, AAA 186, and gateway188. And certainly other network entities and/or combinations of networkentities could be used in various embodiments for carrying out thenetwork-entity functions described herein, as the foregoing list isprovided by way of example and not by way of limitation.

DETAILED DESCRIPTION

A detailed description of illustrative embodiments will now be providedwith reference to the various Figures. Although this descriptionprovides detailed examples of possible implementations, it should benoted that the provided details are intended to be by way of example andin no way limit the scope of the application.

Note that various hardware elements of one or more of the describedembodiments are referred to as “modules” that carry out (i.e., perform,execute, and the like) various functions that are described herein inconnection with the respective modules. As used herein, a moduleincludes hardware (e.g., one or more processors, one or moremicroprocessors, one or more microcontrollers, one or more microchips,one or more application-specific integrated circuits (ASICs), one ormore field programmable gate arrays (FPGAs), one or more memory devices)deemed suitable by those of skill in the relevant art for a givenimplementation. Each described module may also include instructionsexecutable for carrying out the one or more functions described as beingcarried out by the respective module, and it is noted that thoseinstructions could take the form of or include hardware (i.e.,hardwired) instructions, firmware instructions, software instructions,and/or the like, and may be stored in any suitable non-transitorycomputer-readable medium or media, such as commonly referred to as RAM,ROM, etc.

Methods and systems are described for selecting augmented realityinformation without necessarily requiring active participation by humanuser. Various embodiments may address the problem of informationoverload, which is likely to worsen as more AR applications are deployedand more augmented information becomes available.

Methods and systems described herein organize augmented information intospheres (or shapes) of relevance centered on a user. Embodiments firstprovide information about nearby objects, then about farther objects,and even farther objects, and so on, before cycling back to againpresent information about nearer objects. In an exemplary embodiment, inresponse to a significant change in the scene or available augmentedinformation, an exemplary system resets to displaying the sphere nearestto the user. If one or more new objects with augmented informationarrive, an exemplary embodiment may prioritize the display of augmentedinformation to the nearest of such new objects.

Further, methods and systems herein described below are passive in thatthey do not require actions by a user; nor do they require a user'spreferences to be captured. However, the embodiments are modular in thatthey may be readily combined with active, gaze-based, andpreference-oriented approaches (when such information is available), andwhich may be appropriate in some usage scenarios.

Methods and systems are described for new ways of informing user ofinformation that may not be in the user's gaze, without overloading theuser with an extraordinary amount of information. Embodiments describedherein inform the user in a way that is helpful but not intrusive anddoes not require the user to take the initiative. Further, embodimentsdescribed herein inform the user in a way that reduces interference withthe scene and streamlines the presentation of information so that theuser's attention and cognition are not overloaded.

In some embodiments, systems and methods are described for selectingaugmented reality information about one or more real-world objects fordisplay in a user's view comprising, iteratively, receiving one or morepairs of information, each pair comprising (a) information identifying areal-world object, e.g., by its geospatial coordinates, and (b) anassociated AR object (that specifies and renders some associatedaugmented information in a suitable manner, including the name and otherattributes of the real-world object), determining two or more spheres ofrelevance of increasing radii and centered on the user, determining therelevant sphere for current presentation to the user, and displaying ARobjects from among the received real-world objects whose coordinates liewithin the current relevant sphere and within no inner sphere and notdisplaying any other AR objects. In some embodiments, determining therelevant sphere for current presentation comprises setting the relevantsphere as the innermost sphere within which at least one receivedreal-world object lies if there is no prior case or if the user's viewhas changed substantially. In such embodiments, a view changes due tochange of the user's location, the user's head movement, change of theuser's posture. In some embodiments, determining the relevant sphere forcurrent presentation comprises setting the relevant sphere to the spherecentered on the user's eyes within which the nearest newly receivedreal-world object lies if a pair of a real-world object and an AR objectis received that was not present in the pairs received in the previousiteration. In some embodiments, determining the relevant sphere forcurrent presentation comprises setting the relevant sphere to the nextsphere centered on the user's eyes (and cycling back to the innermostsphere if the current sphere is the outermost of the spheres) if therelevant sphere has not changed over a preset duration, e.g., 20seconds. It should be noted that using spheres should not be consideredlimiting. In some embodiments, other geometric shapes may be usedinstead of spheres, such as cubes, ovoids, as well as various otherknown fundamental geometric shapes.

In some embodiments, when a sphere is used for a second or later time(due to the rotation), the AR objects that were previously displayed inthat sphere are not displayed. In some embodiments, spheres are usedwhose diameters increase in some progression, e.g., 5 meters, 10 meters,25 meters. In some embodiments, if all objects previously displayed inthe current sphere move out of that sphere, then the method sets therelevant sphere to the next sphere in the rotation. In some embodiments,other AR objects besides those that lie within the current most relevantsphere may also be displayed.

FIG. 2 is a diagram illustrating a possible architecture, in accordancewith some embodiments. As shown, one or more scene sensors 205 (such asa camera, LIDAR, etc.) provide information about a scene around a user,for example as a point cloud. A location sensor 210 (such as GPS) mayprovide information on the location of the user. A Sensor Aggregator 215receives the scene and location information from the respective sensors,and sends the information to any appropriate (e.g., subscribed)Augmented Reality Applications 220. It also forwards the scene to theAugmented Reality Sphere Constructor 225. In some embodiments, theAugmented Reality applications 220 may be third-party applications whichare independent of a system as disclosed herein. In some embodiments,Augmented Reality applications 220 may receive information about thescene or location (or both), and produce a list of pairs, where eachpair comprises information identifying a real-world object (e.g.,identified by location coordinates) and an AR object that providesinformation about that real-world object and describes how to renderthat information on an AR display.

One component of an exemplary embodiment is the Augmented Reality SphereConstructor 225. Based on a received scene and location (such as from ARapplications 220 and sensor aggregator 215), the AR Sphere Constructor225 may construct two or more spheres (or other range regions) centeredon the user's eyes. The AR Sphere Constructor 225 may receive a list ofpairs from each AR app, as specified above, and assemble these listsinto a master list of pairs. In some embodiments, the AR SphereConstructor 225 is configured to determine the relevant sphere and toselect the objects that lie within the relevant sphere. Subsequently,the AR Sphere Constructor 225 may send the appropriate instructions toan AR Display 230.

In some embodiments, the AR Sphere Constructor 225 operates on anongoing basis. It may maintain a history 235 of its current list ofpairs. If the list of pairs changes, the AR Sphere Constructor 225 mayreevaluate the spheres. In some embodiments, if a new object comes intoview, the AR Sphere Constructor 225 may determine the relevant sphere asa sphere within which the new object lies. If no objects (pairs) arewithin the current sphere, the AR Sphere Constructor 225 may transitionto the next of the currently constructed spheres immediately. If noevent occurs but a sufficient time passes, the AR Sphere Constructor 225may transition to the next of the currently constructed spheres.

In some embodiments, the AR Display 230 presents an augmented view ofthe real world scene to the user. The AR Sphere Constructor 225 may takethe input scene and the selected AR objects to determine what todisplay, and instruct the AR Display 230 accordingly. In someembodiments the AR display 230 may take the form of an AR head-mounteddisplay (HMD). In some embodiments, the HMD may be a transparentdisplay. In some embodiments, the AR HMD may include an opaque screen.In some embodiments, the AR display 230 may take the form of a screen ona smartphone, tablet, or other similar device.

FIG. 3 is flowchart capturing the steps performed in exemplaryembodiments as described in greater detail below. In the discussion ofFIG. 3 and later, “sq” refers to the square function. Notice that thediscussion of the flowchart of FIG. 3 uses the index k to identify asphere (e.g., a spherical shell) that extends from radius r[k] to thenext larger radius r[k+1]. The value k=0 refers to the innermost sphere,which extends to k=1. Similarly, the value k=N−1 refers to the outermostsphere, which extends to infinity since r[N]=infinity. For example, forN=2, there are two spheres, the first from 0 to r[1] and the second fromr[1] to infinity. Similarly, for N=3, there are three spheres: from 0 tor[1]; from r[1] to r[2]; and from r[2] to infinity.

At some point, a system may set a user's frame of reference (305). Forexample, the position of the user may be set to <0,0,0>, with X to theright, Y up, and (negative) Z in the direction of the user's gaze. Thesystem may then receive a list of pairs for <real-world object, ARobject>, each pair being associated with real world coordinates (310).The real-world coordinates for each real-world object may be transformedto the user's frame of reference as <x, y, z> (315).

The system may then determine a number N (N>=2) of spheres (320) havingradii of r[0]<r[1]< . . . <r[N], where r[0]=0 and r[N]=infinity. Thatis, an innermost sphere is k=0 and an outermost sphere is k=N−1. In someembodiments, the system may have predetermined the radii of thesespheres, while in other embodiments the radii may be based on determinedor detected objects.

The system may determine whether it has received a new list of pairs (ora sufficiently different list of pairs from a previous list) (325). Forexample, it may be determined whether a received list has a JaccardIndex below 0.8 with respect to a previous list. In some embodiments,other evaluation techniques for the difference between a received listand a previous list may be used, as known to those of ordinary skill inthe relevant arts, such as but not limited to a Sorenson-Dicecoefficient, an overlap coefficient, and the like.

If the received list is new (or sufficiently different), the system mayselect an innermost nonempty sphere as a relevant sphere (330), bycalculating a smallest k, across all pairs <x, y, z>, such thatsq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]). The system may then displayeach pair of coordinates within the relevant sphere (360), and displayeach pair of coordinates <x, y, z>, such thatsq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]).

If, rather, the received list is determined to not be sufficientlydifferent from a previous list, the system may determine whether thereceived list contains one or more new pairs with respect to theprevious list (335). If the received list does have one or more newpairs, the system may select an innermost sphere with a new object asthe relevant sphere (340), by calculating the smallest k, across the newpairs <x, y, z>, such that sq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]). Thesystem may then display each pair of coordinates within the relevantsphere (360), and display each pair of coordinates <x, y, z>, such thatsq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]).

If it is determined that the received list does not contain any newpairs, the system may determine whether the relevant sphere has beenunchanged for a threshold (or preset) duration (345). If the relevantsphere has been unchanged for the threshold (or preset) duration, thesystem may change the relevant sphere to be the next larger sphere(350), and cycle back to the innermost sphere if at the outermostsphere. In one case, the progression may update the relevant sphere ask=remainder ((k+1)/N). The system may then display each pair ofcoordinates within the updated relevant sphere (360), and display eachpair of coordinates <x, y, z>, such thatsq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]).

For clarity, in an exemplary situation in which nothing changes in theavailable AR information, the spheres may iterate from the smallest tothe largest, and back to the smallest. The update by remainder capturesthis idea. For example, when N=2, a sequence of spheres is described asfollows. An initial value may be 0. A next (second) value may be 1,because the relevant sphere is incremented by 1 from 0 to 1, anddivision of that value by N (2) yields a remainder of 1. A next (third)value may be 0, because the relevant sphere is incremented by 1 from 1to 2, and division of that value by N (2) yields a remainder of 0.

Similarly, when N=3, this update yields a sequence of spheres as 0, 1,2, 0, 1, 2, and so on.

If the relevant sphere has not been unchanged for the threshold (orpreset) duration, the relevant sphere may remain unchanged as k (355).The system may then continue to display each pair of coordinates withinthe relevant sphere (360), and display each pair of coordinates <x, y,z>, such that sq(r[k])<=sq(x)+sq(y)+sq(z)<=sq(r[k+1]).

FIG. 4 is a sequence diagram illustrating how present embodiments may berealized across the various modules. This sequence diagram is anabstract description showing the main entities and their locations in apossible implementation. As shown, an AR application 425 may receivescene and/or location information from a sensor aggregator 410, whichmay receive sensor information from sensors on a user's device, whichmay be an AR HMD, for example. In some embodiments, the sensors mayinclude a GPS to obtain a user's location. In some embodiments, thesensors may include a magnetic compass to provide a viewing directionfor the user. In some embodiments, the sensors may include anaccelerometer that may provide an indication of the user changing theirgaze. In some embodiments, the sensors may include a depth camera todetermine how far away the real-world objects are, so that the AR sphereconstructor may know in which sphere of relevance to display eachreal-world/AR object pair. In some embodiments, the sensors may utilizevarious cameras and 3D sensors such that the AR application may performimage analysis/object recognition to assist in identifying real-worldobjects, as well as finding or obtaining information to display in orwith the AR object.

Using the scene and location information received from the sensoraggregator 410, the AR application 425 may generate pairs of real-worldand AR objects, and provide the pairs to the AR Sphere Constructor 415.In some embodiments, the AR application 425 may identify buildings,landmarks, etc., based on location/direction information provided by thesensor aggregator 410, along with an online map program (e.g., GoogleMaps™, etc.). In some embodiments, the AR application 425 analyzes 3Ddepth information and/or picture/video information to identifyreal-world objects such as cars (and color/type), trees/flowers (basedon shape of leaf, for example), insects/animals, fountains/statues,among various other real-world objects that may be found interesting tothe user. In some embodiments, AR applications 425 may be pre-configuredby a user to show things of interest (e.g., trees/animals/flowers for anature enthusiast, all nearby restaurants for a food enthusiast, etc.).The AR application 425 may access additional types of databases foridentifying various types of real-world objects, and the above examplesshould not be considered limiting.

The AR Sphere Constructor 415 may further receive previous lists ofpairs (if any) from a history/memory device 420, and provides thehistory/memory device 420 with the pairs the AR Sphere Constructor 415received from the AR application 425. The AR Sphere Constructor 415 thenconstructs two or more spheres of relevance, and selects a relevantsphere based on (i) changes to the list of pairs and (ii) a passage oftime. Once a relevant sphere has been selected, AR objects within therelevant sphere are selected, and the scene with the augmented ARobjects is displayed on the user's AR display 405 (such as an HMD).

In some embodiments, spheres may be interpreted as centered on theuser's eyes, and constructed by specifying one or more radii betweenzero and infinity. Accordingly, in some embodiments, the first sphere isbetween zero and the smallest radius, and the last sphere (or, rather,spherical shell) is between the largest radius and infinity. The radiimay follow a simple pattern such as 10 meters, 20 meters, and so on (or5, 10, 15, . . . , and/or the like). In some cases, the radii may havelogarithmic or exponential progression, or the like.

At various stages, the system may operate on pairs of a real-worldobject (specified via its coordinates) and an AR object (providinginformation about that real-world object). In some embodiments, a methodincludes comparing a previous list of pairs to a current list of pairsby comparing each pair in the previous list with each pair in thecurrent list. For example, a previous list's pair may have coordinates<xp, yp, zp>, and a current list's pair may have coordinates <xc, yc,zc>. In one embodiment, the two pairs may be determined as being thesame if two tests return true. First, the system may determine if thereal-world coordinates of the two pairs (from previous and currentlists, respectively) are approximately equal by verifying if the sum ofsq(xc−xp)+sq(yc−yp)+sq(zc−zp) is below some preset threshold (e.g., 0.5meters, 0.2 meters, 0.3 meters, 0.1 meters, 1 meter, etc.). Second, thesystem may determine if the AR object information of each pair is“equal” by verifying if the AR object information components of the twopairs are identical.

In some embodiments, the system may compute a Jaccard Index using theabove computations of agreement between the pairs that occur in theprevious and current lists (or some other computations of agreement, asknown to those of ordinary skill in the relevant art). The Jaccard Indexis used to measure dissimilarity between sample sets. Thus, if theJaccard Index is below some preset threshold (e.g., 0.8, 0.9, 0.75,etc.), then the lists of pairs are treated as having changedsignificantly. In some embodiments, other evaluation techniques for thedifference between a received list and a previous list may be used, asknown to those of ordinary skill in the relevant arts, such as but notlimited to a Sorenson-Dice coefficient, an overlap coefficient, and thelike.

In some embodiments, two pairs are considered as being “identical” evenif there is some variation in the measurement of the coordinates (due toinstrument error or slight movements of the user's head and body).However, in some embodiments, if the AR information (metadata) changeseven slightly that can mean either that a new real-world object hasmoved into the position previously occupied by some other object or thatan AR application has found it appropriate to produce new informationabout the same real-world object. In either case, this is a change thatmay be valuable to the user.

In some embodiments, the method is passive in that it continues even asthe user is not taking the initiative. Such embodiments may help theuser gradually become familiar with his or her surroundings in a casual,intuitive manner. Further, embodiments described above may track changesto the user's scene and respond appropriately. Some embodiments may beinstantiated on top of existing AR applications and displays. Indeployed products, embodiments described above may be combined withother techniques, such as receiving prioritized information from ARapplications and considering the user's preferences across the availableAR information.

In some embodiments, the AR display 405 and sensor aggregator 410 may becomponents of a user's device. In some embodiments, the AR sphereconstructor 415 may be a component of a user's device, or may be acloud-based component or otherwise remote and in communication with theuser's device. In some embodiments, the history 420 of pair lists may bemaintained in a memory of the user's device, or in a cloud or remotedatabase/memory. In some embodiments, the AR application(s) 425 may becloud based, while in other embodiments they may be implemented on theuser's device.

In some embodiments, a system may selectively present augmented realityinformation to a user. The system may detect or otherwise determine aplurality of real-world objects in a field of view of the user. Thesystem may then select from the plurality of real-world objects a firstgroup of at least one real-world object less than a first distance fromthe user in the field of view. The first distance may be a presetdistance (e.g., 5 ft, 10 ft, etc.), or may selected by the system basedon the locations of each of the detected plurality of real-worldobjects. The system may also select from the plurality of real-worldobjects a second group of at least one real-world object in the field ofview of the user between the first distance and a second distance fromthe user, where the second distance is greater than the first distance.In some embodiments, additional groups of objects for regions extendingto further distances may also be selected (e.g., a third group betweenthe second distance and a greater third distance, etc.). In someembodiments, each of the distances may be chosen by the system based onthe locations of the real-world objects. The system may augment thefirst group of real-world objects at a first time. In some embodiments,the first time is a time period of a preset duration (e.g., 20 seconds,30 seconds, etc.). In some embodiments, the time period may be chosen bythe system based on the number of objects in the first group (orsimilarly chosen by the system for any group based on the size of thegroup). The system may then augment the second group of real-worldobjects at a second time after the first time. In some embodiments, thesecond time may be a time period of a preset duration, or a time periodof a duration chosen by the system. In some embodiments, rather thanhaving time periods, the system may augment a group of objects at theirassociated time, and stop augmenting the group prior to augmentinganother group of objects. If there are more than two groups of objects,the system may progressively augment each group at a subsequent time.

In some embodiments, the system may, after the second time, augment thefirst group of objects at a third time. The system may also augment thesecond group of objects at a fourth time after the third time. Any othergroups may also progressively be augmented again at subsequent times. Insome embodiments, the augmenting of groups may be performed cyclically,with each group augmented at different times in each of a plurality ofcycles. In some cases, the order of groups in each cycle may be thesame, and in others the order may be different (such as depending ondetection of additional real-world objects).

In some embodiments, the system may detect one or more additionalreal-world objects after grouping the originally detected objects. Insome cases, such as for an additional object falling within the firstdistance but detected prior to augmenting the second group of objects,the system may augment the additional object at a third time after thefirst time but before the second time. After augmenting the additionalobject, the system may proceed to augmenting the second group.Similarly, if an additional object were detected after augmentation of asecond group but before augmentation of a third group between the seconddistance and a greater third distance, and the additional object fallswithin the range associated with the first or second group, the systemmay augment the additional object after the second group but before thethird group. If an additional object falls within a group that has notyet been augmented, it may simply be added to the appropriate group andaugmented at the same time as the rest of that group.

In some embodiments, such as when a user turns their head, the systemmay detect the change in field of view. The system may then detect orotherwise determine an updated plurality of real-world objects in thechanged field of view of the user. The system may then select groups ofobjects for each distance range, such as a third group within the firstdistance and a fourth group between the first and second distances. Thesystem may then progressively augment these groups of objects, such asby augmenting the third group at a third time, and then augmenting thefourth group at a fourth time after the third time.

In some embodiments, the user's field of view may be determined by thesystem from one or more sensor measurements (such as GPS, compass,etc.), by an image detection process, or the like.

FIGS. 5A-5D illustrate a progression of spheres, in accordance with someembodiments. Each diagram shows a user 503 with concentric spheres: 505,510 and 515. The chevron 520 indicates the view angle (i.e., the smallerangle of the chevron) of the user 503. The received AR objects arescattered about the spheres. As shown in FIG. 5A, there may be ARobjects behind the user 503, because some AR apps may work or operatebased on location alone. An AR object 525 in the first sphere 505 andwithin the user's view angle is displayed or otherwise presented, whileother AR objects are not.

As shown in FIG. 5A, an initial view displays to the user 503 ARinformation about visible objects 525 in the nearest sphere 505 withinthe user's viewing angle 520. After some duration (e.g., 20 seconds),the device may show (or present) AR information about visible objects527 in the second sphere 510 within the user's viewing angle 520, asshown in FIG. 5B. After another duration, an object of interest 526 maymove to a location of the innermost sphere 505 within the user's viewingangle 520, and, rather than progressing to the outermost sphere 515, theAR device may reset (or return) to the innermost sphere 505 and displayboth objects 525 and 526, as shown in FIG. 5C. If at some point the user503 turns around or otherwise alters their viewing angle 520, the scenevisible to the user 503 may change drastically, and the AR device mayreset (or return) to the innermost sphere 505, as shown in FIG. 5D, todisplay or otherwise present AR information for an object 528 that isnow within the user's viewing angle 520 in the innermost sphere 505.

In an exemplary scenario, a user is spending an afternoon in a townsquare. An AR device first provides the user with information about thesquare and shops that line the square. An example of this is illustratedby FIG. 6A. After a while, as illustrated by FIG. 6B, the AR devicebrings forth information about some nearby buildings in a second sphere.A short while later, as illustrated by FIG. 6C, the AR device points outthat a landmark is a bit farther away toward the user's right in thethird sphere.

Suddenly, as illustrated by FIG. 6D, a small troupe of dancers come toan empty part of the square near where the user is sitting and begin toset up their portable dance tiles and their musical instruments. Just asthe user's attention comes back to the square, the AR device refocuseson the nearby objects and tells her about this dance troupe, which isapparently one of the longest running street performers group. Thedancers set up and settle down for some lemonade before theirperformance. The AR device then tells the user about some objectsfarther in the distance.

The user's daughter suddenly calls the user's attention with a cry of aseeing a celebrity at a nearby table. The user turns around to look atthe table, and the AR device resets to the new scene. The AR devicedetects it is not the celebrity, just a person with a similarappearance. Later, a procession with a marching band goes by somedistance away. Even though it is not in the inner three spheres renderedby the AR device, because it causes a change with a new pair of <band,AR info about the band>, the AR device selects the sphere that containsthe band.

As time passes, the scene settles down. No new real-world objects areappearing in the scene, and the AR device displays AR information forthe real-world objects in a fourth sphere, as illustrated by FIG. 6E. Insome embodiments, the AR information for objects in the fourth sphere isavailable after AR information for real-world objects in the firstthrough third spheres has already been shown.

FIG. 7 illustrates a progression of augmenting real-world objects, inaccordance with some embodiments. In some embodiments, a method forselectively presenting augmented reality information includes detectinga direction-of-view of a user 705, determining or selecting at least onereal-world object 710 less than a first predetermined distance D1 fromthe user in a field of view 715 associated with the determineddirection-of-view of the user 705, determining or selecting at least onereal-world object 720 between the first predetermined distance D1 of theuser 705 and a second predetermined distance D2 from the user 705,augmenting the at least one real-world object 710 less than the firstpredetermined distance D1 for a first time period t₁, and augmenting theat least one real-world object 720 between the first predetermineddistance D1 of the user and a second predetermined distance D2 for asecond time period t₂ subsequent to the first time period t₁. As shown,a first sphere of relevance 730 may be defined as the region between theuser and the first predetermined distance D1, and a second sphere ofrelevance 740 may be defined as the region between the firstpredetermined distance D1 and the second predetermined distance D2.

In some embodiments, rather than detecting a direction of view of theuser, a field of view of the user may be known or obtained, andreal-world objects in the field of view selected.

In some embodiments, a second time period may be triggered or initiatedby an input from the user, such as a voice command (“go on”, etc.),selection input (button press, etc.), or signal of intent from the user.For example, this may permit a user to take their time to evaluate theaugmented objects. In some embodiments, there may be a transition periodbetween the first and second time periods, such that the time periodsmay partially overlap. In other embodiments, the first and second timeperiods do not overlap.

In some embodiments, a user may select or otherwise adjust thepredetermined distances used by the systems and methods herein. Forexample, a user may adjust a first predetermined distance to be smallerfor a scenario where many objects are going to be near the user, oradjust the distance to be greater for a scenario where objects are farapart. In some embodiments, the second predetermined distance (or othersafter the first distance) may be adjusted by the user after augmentationof objects within the first predetermined distance. For example, theuser may see an object some distance away from the augmented objects(e.g., something far off that may fall into a third, fourth, or the likedistance or sphere), and provide an input to the system (such as a voicecommand, button input, slider movement, etc.) to adjust the range wherea subsequent grouping of real-world objects will be augmented.

In some embodiments, the predetermined distances may be determined bythe system in response to context aware analysis, such as evaluation ofthe number and location of real-world objects around the user.

In one embodiment, there is a method for selectively presentingaugmented reality information, comprising: detecting a direction-of-viewof a user; determining at least one real-world object less than a firstpredetermined distance from the user in a field of view associated withthe determined direction-of-view of the user; determining at least onereal-world object between the first predetermined distance of the userand a second predetermined distance from the user; augmenting the atleast one real-world object less than the first predetermined distancefor a first time period; and augmenting the at least one real-worldobject between the first predetermined distance of the user and a secondpredetermined distance for a second time period subsequent to the firsttime period. The method may include wherein augmenting the at least onereal-world objects comprises displaying an augmented reality (AR) objectassociated with a respective real-world object. The method may alsoinclude wherein the AR object is displayed in a sphere of relevancedetermined by a distance of the real-world object from the user. Themethod may further comprise augmenting the third real-world object. Themethod may also include wherein the third real-world object is withinthe first predetermined distance, and augmented at a third time periodsubsequent to the second time period. The method may include wherein thefirst time period and second time periods are fixed durations. Themethod may further comprise detecting a change in direction-of-view ofthe user; and augmenting real-world objects within the firstpre-determined distance. The method may include wherein the first andsecond predetermined distances define first and second spheres ofrelevance, respectively.

In one embodiment, there is a method for selecting augmented reality(AR) information about one or more real-world objects for display in auser's view, the method comprising: receiving one or more pairs ofinformation, each pair comprising (i) information identifying areal-world object by geospatial coordinates, and (ii) an associated ARobject including information about the real-world object; determiningtwo or more spheres of relevance, each sphere of relevance centered onthe user, and having a corresponding radius; and determining a relevantsphere for presentation to the user, and responsively displaying ARobjects from among the received real-world objects whose coordinates liewithin the current relevant sphere and within no inner sphere, and notdisplaying any other AR objects. The method may include whereindetermining the relevant sphere comprises determining there is no priorcase or if the user's view has changed substantially, and responsivelysetting the relevant sphere to an innermost sphere within which at leastone real-world object lies. The method may also include whereindetermining the relevant sphere comprises determining a new pair ofreal-world object and AR object is received that was not present in aprevious iteration, and responsively setting the relevant sphere to asphere associated with the new real-world object. The method may alsoinclude wherein determining the relevant sphere comprises determining acurrent relevant sphere has not changed for a predetermined duration,and responsively setting the relevant sphere to the next sphere ofrelevance in a predetermined iteration sequence. The method may alsoinclude wherein the predetermined iteration sequence goes frominner-most spheres of relevance to outer-most spheres of relevance. Themethod may also include wherein if the current relevant sphere is theouter-most sphere of relevance, the subsequent relevant sphere is theinner-most sphere of relevance.

In one embodiment, there is a method comprising: receiving one or morepairs of information, each pair comprising (i) information identifying areal-world object and (ii) an associated augmented reality (AR) objectincluding information about the real-world object; selecting a relevantsphere from a plurality of spheres of relevance, each sphere ofrelevance having an associated radius; and displaying AR objectsassociated with real-world objects located within the selected relevantsphere to a user. The method may include wherein selecting the relevantsphere comprises: identifying a substantial change in the user's gaze,and selecting an inner-most sphere of the plurality of spheres ofrelevance as the relevant sphere. The method may also include whereinthe inner-most sphere is the inner-most sphere including at least onepair of information. The method may include wherein selecting therelevant sphere comprises identifying a new pair of information notpresent in a previous iteration, and responsively setting the relevantsphere to the sphere of relevance associated with the new pair ofinformation. The method may include wherein selecting the relevantsphere comprises identifying no new pair of information has beenreceived for a predetermined duration, and responsively setting therelevant sphere as a next sphere in a predetermined iteration sequence.The method may also include wherein the predetermined iteration sequencecyclically iterates from spheres of relevance in increasing radii. Themethod may also include wherein the predetermined iteration sequencecyclically iterates from spheres of relevance in decreasing radii. Themethod may include wherein each pair of information includes3-dimensional coordinates <xc, yc, zc> for the real-world object and theAR object.

In one embodiment, there is a method comprising: setting a userframe-of-reference; receiving a list of pairs, each pair including 3-Dcoordinates of a real-world object and an AR object; transforming eachpair into the user frame-of-reference; determining a plurality ofspheres of relevance, each sphere having an associated radius;determining a relevant sphere of the plurality of spheres of relevanceby: determining if a new list of pairs is substantially different than aprevious iteration, and if so setting the relevant sphere to aninner-most non-empty sphere of relevance; determining if the receivedlist includes one or more new pairs, and if so setting the relevantsphere to the innermost sphere of relevance with associated with atleast one of the one or more new pairs; determining if the relevantsphere is unchanged for a preset duration, and if so setting the spherewith the next largest radius as the relevant sphere; and displaying ARobjects associated with real-world objects within the determinedrelevant sphere. The method may include wherein determining that the newlist of pairs are substantially different comprises calculating aJaccard index. The method may also include wherein a Jaccard index ofless than 0.8 determines that the new list of pairs is substantiallydifferent from the current list of pairs.

In one embodiment, there is an apparatus comprising: a) an augmentedreality (AR) application configured to receive location information froma sensor aggregator, and to responsively generate one or more pairs ofreal-world and AR objects; b) an AR sphere constructor configured to:receive (i) the one or more pairs from the AR application and (ii) aprevious list of pairs from a history device; provide the one or morepairs received from the AR application to the history device; constructone or more spheres of relevance; set a relevant sphere based on changesto the receive list of pairs; and select AR objects associated withreal-world objects within the relevant sphere; and c) an AR displayconfigured to display a scene including the selected AR objects. Theapparatus may include wherein the AR display comprises an ARhead-mounted display (HMD). The apparatus may include wherein the ARdisplay comprises a screen.

The apparatus may also include wherein the screen is transparent. Theapparatus may also include wherein the screen is opaque and displays avideo of the user's environment. The apparatus may include wherein thesensor aggregator is configured to obtain sensor information from one ormore sensors. The apparatus may also include wherein the one or moresensors comprises a depth camera. The apparatus may also include whereinthe one or more sensors comprises a global positioning system (GPS). Theapparatus may also include wherein the one or more sensors comprises acompass. The apparatus may also include wherein the one or more sensorscomprises a depth sensor. The apparatus may also include wherein the oneor more sensors comprises a camera.

In one embodiment, there is a method comprising: receiving a list ofinformation pairs, each information pair comprising information about areal-world object and an associated augmented reality (AR) object;generating one or more spheres of relevance; and selecting a sphere ofrelevance based at least in part on one of the received list ofinformation pairs. The method may further comprise displaying a sceneincluding the real-world object and associated AR object of the one ofthe received list of information pairs. The method may include whereinselecting the sphere of relevance comprises identifying the receivedlist of information pairs is different than a previously received listof information pairs. The method may also include wherein a Jaccardindex is used to compare the received list of information pairs and thepreviously received list of information pairs. The method may alsoinclude wherein selecting the sphere of relevance comprises selecting aninner-most sphere of relevance associated with one of the pairs in thereceived list of information pairs. The method may include whereinselecting the sphere of relevance comprises identifying that thereceived list of information pairs is within a threshold of being thesame as a previously received list of information pairs, and furthercomprise selecting the next sphere of relevance in a predeterminediteration sequence. The method may also include wherein each sphere ofrelevance has an associated radius. The method may also include whereinthe predetermined iteration sequence cyclically selects spheres ofrelevance in an increasing-radius direction. The method may also includewherein the predetermined iteration sequence cyclically selects spheresof relevance in a decreasing-radius direction.

In one embodiment, there is a method comprising: identifying a pluralityof real-world objects having respective associated annotationinformation and respective distances from a user; cyclically displayingto the user, on an augmented reality display, at different times: (i) afirst subset of annotations for those real-world objects having a firstrange of distances, and (ii) at least a second subset of annotations forthose real-world objects having a second range of distances that doesnot overlap with the first range. The method may include wherein thecyclic display of the annotations comprises, in each cycle, displayingsubsets of annotations in order of increasing distances. The method mayfurther comprise, in response to detection of a new real-world objecthaving an associated new-object annotation and a distance from the userwithin an Nth range of distances, interrupting the cyclic display orderto next display the Nth subset of annotations including the new-objectannotation. The method may further comprise, in response to asubstantial change of viewing direction by the user, interrupting thecyclic display order to next display the subset of annotations havingthe lowest distance.

In one embodiment, there is a method for selecting augmented realityinformation about one or more real-world objects for display in a user'sview comprising, iteratively, the method comprising: a) receiving one ormore pairs of information, each pair comprising (i) information about areal-world object identified by its geospatial coordinates and (ii) anassociated AR object, the associated AR object specifying and renderingassociated augmented information; b) determining two or more shapes ofrelevance of increasing size and centered on the user; c) determining arelevant shape for current presentation to the user, wherein determiningcomprises: if there is no prior case or if the user's view has changedsubstantially, then setting the relevant shape as the innermost shapewithin which at least one received real-world object lies; otherwise, ifa pair of real-world object and AR object is received that was notpresent in the pairs received in the previous iteration, then settingthe relevant shape to the shape centered on the user's eyes associatedwith the nearest newly received real-world object; otherwise, if therelevant shape has not changed over a preset duration, then setting therelevant shape to the next larges shape centered on the user's eyes, andcycling back to the innermost shape if the current shape is theoutermost of the shapes; and d) displaying AR objects associated withthe received real-world objects, the AR objects having coordinateslaying (i) within the current relevant shape and (ii) not within anyinner shapes, and not displaying any AR objects having coordinatesoutside the relevant sphere. The method may include wherein theassociated augmented information includes a name and other attributes ofthe real-world object. The method may include wherein the shapes ofrelevance are spheres. The method may also include wherein spheres whosediameters increase in some progression determined by radius. The methodmay also include wherein each consecutive sphere of relevance has aradius of 5 meters, 10 meters, and 25 meters. The method may includewherein the shapes of relevance are cubes. The method may includewherein the user's view changing comprises a change of the user'slocation. The method may include wherein the user's view changingcomprises user's head movement. The method may include wherein theuser's view changing comprises a change of the user's posture. Themethod may include wherein if all objects previously displayed in thecurrent shape move out of that shape, then the method sets the relevantshape to the next shape in the rotation. The method may further comprisedisplaying at least one other AR objects besides those that lie withinthe current most relevant shape.

Although features and elements are described above in particularcombinations, one of ordinary skill in the art will appreciate that eachfeature or element can be used alone or in any combination with theother features and elements. In addition, the methods described hereinmay be implemented in a computer program, software, or firmwareincorporated in a computer-readable medium for execution by a computeror processor. Examples of computer-readable storage media include, butare not limited to, a read only memory (ROM), a random access memory(RAM), a register, cache memory, semiconductor memory devices, magneticmedia such as internal hard disks and removable disks, magneto-opticalmedia, and optical media such as CD-ROM disks, and digital versatiledisks (DVDs). A processor in association with software may be used toimplement a radio frequency transceiver for use in a WTRU, UE, terminal,base station, RNC, or any host computer.

What is claimed:
 1. A method for selectively presenting augmented reality (AR) information, comprising: detecting a plurality of real-world objects in a field of view of a user; selecting from the plurality of real-world objects a first group of at least one real-world object within a first shape having a first size; selecting from the plurality of real-world objects a second group of at least one real-world object outside the first shape and within a second shape having a second size larger than the first size; augmenting the first group of real-world objects at a first time; and augmenting the second group of real-world objects at a second time after the first time; wherein the first size and the second size are selected based at least in part on the locations of each of the detected plurality of real-world objects.
 2. The method of claim 1, further comprising stopping augmentation of the first group of real-world objects prior to the second time.
 3. The method of claim 1, further comprising: selecting from the plurality of real-world objects a third group of at least one real-world object outside the second shape; and augmenting the third group of real-world objects at a third time after the second time.
 4. The method of claim 1, wherein the first shape and the second shape are cubes.
 5. The method of claim 1, wherein the first shape and the second shape are ovoids.
 6. The method of claim 1, wherein the first shape and the second shape are spheres.
 7. The method of claim 1, wherein the first shape and the second shape are substantially centered on the user.
 8. An apparatus comprising a processor configured to perform: detecting a plurality of real-world objects in a field of view of a user; selecting from the plurality of real-world objects a first group of at least one real-world object within a first shape having a first size; selecting from the plurality of real-world objects a second group of at least one real-world object outside the first shape and within a second shape having a second size larger than the first size; augmenting the first group of real-world objects at a first time; and augmenting the second group of real-world objects at a second time after the first time; wherein the first size and the second size are selected based at least in part on the locations of each of the detected plurality of real-world objects.
 9. The apparatus of claim 8, wherein the processor is further configured to stop augmentation of the first group of real-world objects prior to the second time.
 10. The apparatus of claim 8, wherein the processor is further configured to perform: selecting from the plurality of real-world objects a third group of at least one real-world object outside the second shape; and augmenting the third group of real-world objects at a third time after the second time.
 11. The apparatus of claim 8, wherein the first shape and the second shape are cubes.
 12. The apparatus of claim 8, wherein the first shape and the second shape are ovoids.
 13. The apparatus of claim 8, wherein the first shape and the second shape are substantially centered on the user.
 14. A method for selectively presenting augmented reality (AR) information, comprising: detecting a plurality of real-world objects in a field of view of a user; selecting from the plurality of real-world objects a first group of at least one real-world object less than a first distance from the user in the field of view; selecting from the plurality of real-world objects a second group of at least one real-world object in the field of view of the user between the first distance and a second distance from the user greater than the first distance; augmenting the first group of real-world objects at a first time; and augmenting the second group of real-world objects at a second time after the first time; wherein the augmenting of at least the first and second group of real-world objects is performed cyclically, each group being augmented at different times in each of a plurality of cycles.
 15. The method of claim 14, further comprising stopping augmentation of the first group of real-world objects prior to the second time.
 16. The method of claim 14, wherein augmenting a real-world object comprises displaying an AR object associated with the real-world object.
 17. The method of claim 14, further comprising: selecting from the plurality of real-world objects a third group of at least one real-world object in the field of view of the user beyond the second distance; and augmenting the third group of real-world objects at a third time after the second time.
 18. The method of claim 14, wherein the first time has a first preset duration and the second time has a second preset duration.
 19. The method of claim 14, further comprising: detecting, prior to augmenting the second group of real-world objects, an additional real-world object in the field of view of the user; and augmenting the additional real-world object at a third time.
 20. The method of claim 14, further comprising: detecting a change in the field of view of the user; detecting an updated plurality of real-world objects in the changed field of view of the user; selecting from the updated plurality of real-world objects a third group of at least one real-world object less than the first distance from the user in the changed field of view; selecting from the updated plurality of real-world objects a fourth group of at least one real-world object in the field of view of the user between the first distance and the second distance; augmenting the third group of real-world objects at a third time; and augmenting the fourth group of real-world objects at a fourth time after the third time. 